A framework for the Assessment of Research and its impacts

This paper proposes a framework for the development of models for the assessment of research activities and their impacts. It distinguishes three dimensions: theory, methodology and data, each of which is further characterized by three main building blocks: education, research and innovation (theory); efficiency, effectiveness and impact (methodology); and availability, interoperability and “unit free” property (data). The different dimensions and their nine constituent building blocks are attributes of an overarching concept, denoted as “quality”. Three additional quality attributes are identified as implementation factors (tailorability, transparency and openness) and three “enabling” conditions (convergence, mixed methods and knowledge infrastructures) complete the framework. The paper illustrates the complexity of the evaluation describing the generalized “implementation problem” in research assessment, according to the proposed framework. A framework is required to develop models of metrics. Models of metrics are necessary to assess the meaning, validity and robustness of metrics. The proposed framework can be a useful reference for the development of the ethics of research evaluation. Three examples of application, as well as further directions for future research are provided.